This invention is in the fields of electronic integrated circuits, digital video and digital signal processing, wireless devices, electronic processors and systems, and remote computing.
An important problem currently believed to be suffered and unsolved by these diverse devices or applications is how to handle video and graphics when video portions or clips can be or are likely to accompany or be embedded in graphics, or vice versa. Remote computing applications such as PC gaming, thin client, and netbook have been growing in the computing space as more compute power is being done or will be done in the upstream, for example cloud computing. Also, video and graphics may occur alternately in succession or be combined on a screen during control of several computers by a particular keyboard, video display, and mouse (KVM). Some systems work oppositely by having several keyboards, video displays, and mice controlling computers that are fewer in number than the keyboards, video displays, and mice. The proliferation of social media and all types of web video and graphics applications challenge the art to handle video and graphics in succession and together.
Graphics with embedded video (and video with embedded graphics) constitutes a dilemma, first, because using a “lossy” video compression process like H.264 and its cousins is very likely to degrade quality of the graphics portion of the screen. On the other hand, applying or using a lossless compression process to preserve the graphics portion may cause received video to exhibit enough video motion artifacts to detract from user experience. “Lossy” in the image compression/decompression art means that some content of an original image supplied for compression becomes somewhat altered in the output from decompression as compared with the original image, due to the compression/decompression processing. “Lossless” means that the original image is reproduced without being altered. A given image compression/decompression process may be lossless with respect to one type of image and lossy with respect to another type of image.
Image compression, even if it involves some loss of information end-to-end by the time of eventual display, is an important technology because it very substantially reduces the amount or rate of information that needs to be employed to convey an image from its source to a destination compared to the more modest attendant loss of information end-to-end due to the compression/decompression process itself. Image compression can also conserve or reduce memory for greater economy and/or more responsive device performance, impact on overall energy consumption, and better user experience. Different methods of image compression and decompression process the information differently, have different objectives and can have different properties and effects on images passing through them. For some background on some known systems with quite different objectives and techniques, see US Patent Application Publication 20110078532 “Method and System for Low Latency Transfer Protocol,” dated Mar. 31, 2011; US Patent Application Publication 20100135397 “Video Encoding Apparatus and Video Encoding Method,” dated Jun. 3, 2010; and US Patent Application Publication 20100316126, “Motion-Based Dynamic Resolution Multiple Bit Rate Video Encoding” dated Dec. 16, 2010.
Computers are very powerful now and will become more powerful in the future. This makes remote desktop applications very practical from the viewpoint of computing power. An important challenge or question facing the art is how to provide devices and processes that can deliver high quality graphics and video in diverse instances wherein they occur simultaneously or in rapid succession. The art is challenged to deliver same in the midst of a plethora of mutually-contradictory user and system demands, such as for high image quality regardless of the type of image or of portions of a composite image, demands for low latency, rapid response, bandwidth conservation, low power and energy consumption and effective power management, and for long battery life in battery-powered devices.
Accordingly, some remarkable technological departures for devices, systems, and processes for handling such multiple kinds of content would be most desirable.